Equivalence between LINE and Matrix Factorization

LINE [1], as an efficient network embedding method, has shown its effectiveness in dealing with large-scale undirected, directed, and/or weighted networks. Particularly, it proposes to preserve both the local structure (represented by First-order Proximity) and global structure (represented by Second-order Proximity) of the network. In this study, we prove that LINE with these two proximities (LINE(1st) and LINE(2nd)) are actually factoring two different matrices separately. Specifically, LINE(1st) is factoring a matrix M , whose entries are the doubled Pointwise Mutual Information (PMI) of vertex pairs in undirected networks, shifted by a constant. LINE(2nd) is factoring a matrix M , whose entries are the PMI of vertex and context pairs in directed networks, shifted by a constant. We hope this finding would provide a basis for further extensions and generalizations of LINE. 1 Notation and Definition Given a network G = (V , E), where each edge e ∈ E is an ordered pair e = (vi, vj) and has an associated weight wij > 0. In directed networks, the in-degree and out-degree of vertex vi are denoted as deg (vi) and deg (vi) respectively. In addition, in undirected networks, the degree of vertex vi is denoted as deg(vi). The first-order proximity [1] characterizes the local structure similarity between vertices. More specifically, if (vi, vj) ∈ E , wij indicates the first-order proximity between vi and vj , otherwise their first-order proximity is 0. The second-order proximity [1] characterizes the global structure similarity between vertices. Mathematically, let pi = (wi1, ..., wi|V|) represent the first-order proximity between vi and the other vertices, then the second-order proximity between vi and vj is characterized by the similarity between pi and pj . To simultaneously preserve these two proximities, Tang et al. [1] train the LINE model which preserves the first-order proximity (denoted as LINE(1st)) Corresponding author: Zheng Wang.